Visual Laughter Synthesis: Initial Approaches

Visual laughter synthesis is a challenging task that was only rarely explored and empirical investigations are scarce. For the purpose of building a virtual agent able to laugh naturally we exploit different animation techniques such as a procedural animation or based on motion capture and we apply them to visual laughter synthesis. At the moment we focus on three approaches: procedural animation based on manual annotation of facial behavior; motion capture driven animation and animation generated from automatic facial movements detection. For the purpose of this study we use the Greta agent (Niewiadomski et al., 2011) that can be driven by both high-level anatomically inspired facial behavior description based on the Facial Action Coding System (FACS; Ekman, et al., 1978) or low-level facial animation parameterization (FAPs) that is a part of MPEG-4 standard (Ostermann, 2002) for facial animation. We also use two video corpora: AVLC database (Urbain et al., 2011) containing mocap, video and audio data of 24 subjects showing spontaneous amusement laugh responses and Queen's University Belfast's dataset of same sex interaction dyads during the watching of funny stimuli. We present all approaches in detail. Manual annotation of action units. FACS is a comprehensive anatomically based system for measuring all visually discernible facial movement. It describes all distinguishable facial activity on the basis of 44 unique Action Units (AUs), as well as several categories for head and eye positions/movements and miscellaneous actions. Using FACS and viewing digital-recorded facial behavior at frame rate and in slow motion, certified FACS coders are able to distinguish and code all visually discernible facial expressions. Utilizing this technique, a selection of twenty pre-recorded, laboratory stimulated, laughter events were coded. These codes were then used to model the facial behavior on the Greta agent which is able to display any configuration of AUs. For 3 virtual characters single AUs were defined and validated by certified FACS coders under the constraints of the technology. A Behavior Markup Language (BML) implemented in Greta permits the control of each AU of the agent (its duration and intensity) independently. The animation of any AU is linearly interpolated according to Attack-Decay-Sustain-Release model (Ekman, et al., 1978). Next, the symbolic intensity values are converted to low-level facial animation parameters (FAP) which, finally, are used to deform a mesh of the virtual model. We also developed a tool that automatically converts manual annotation files created with Noldus Observer XT, a commercial tool …